BLEU SCORE

Answering Naturally: Factoid to Full length Answer Generation (D19-54)

Challenge:	MT has seen significant advances in recent years, but the representation of African languages in MT systems is underrepresented due to linguistic complexities and limited resources.
Approach:	They propose a first robust parallel French-Zarma corpus and a glossary for MT that contains 61,085 sentences in Zarma and 42,789 in French.
Outcome:	The proposed model improves the representation of the Zarma language, a dialect of Songhay, spoken by over 5 million people across Niger and neighboring countries.

Copied to clipboard

Vaishali Pal, Manish Shrivastava, Irshad Bhat

Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance (2024.acl-short)

Challenge:	Factoid question answering systems extract answers for a question from passages, which are usually short spans of text . but, these spans would result in an unnatural reading experience in a conversational system . a pointer generator based full-length answer generator can be used with most QA systems .
Approach:	They propose a pointer generator based full-length answer generator which can be used with most QA systems.
Outcome:	The proposed system generates full length answer without relying on passage from which it was extracted.

Copied to clipboard

Yewei Song, Cedric Lothritz, Xunzhu Tang, Tegawendé Bissyandé, Jacques Klein

Syntactic Text Processing

Challenge:	Abstract Syntax Tree (AST) editing distance is a new evaluation metric for code generation tasks.
Approach:	They propose, optimize, and publish an enhanced version of Tree Similarity of Edit Distance (TSED) based on AST editing distance and prompt-based GPT similarity scores.
Outcome:	The proposed metric is an enhanced version of Tree Similarity of Edit Distance (TSED) it is compared to BLEU score, execution match, and Jaccard similarity across languages.

Context-aware Neural Machine Translation with Coreference Information (D19-65)

Copied to clipboard

Takumi Ohtani, Hidetaka Kamigaito, Masaaki Nagata, Manabu Okumura

English to Hindi Multi-modal Neural Machine Translation and Hindi Image Captioning (D19-52)

Challenge:	Existing models for translating a sentence in a text do not consider coreference relations provided within the text.
Approach:	They propose a graph-based encoder which can consider coreference relations provided within the text explicitly.
Outcome:	The proposed model improves on the previous approach by 0.9 points on the BLEU score . the graph-based encoder can handle a longer text well, compared with the previous model .

Copied to clipboard

Sahinur Rahman Laskar, Rohit Pratap Singh, Partha Pakray, Sivaji Bandyopadhyay

Making Asynchronous Stochastic Gradient Descent Work for Transformers (D19-56)

Challenge:	Multi-modal translation is an emerging task of the MT community, where visual features of image combine with textual features of parallel source-target text to translate sentences.
Approach:	They propose to use convolutional neural net-works and visual geometry to extract image features and attention-based Neural MachineTranslation (NMT) system for translation.
Outcome:	The proposed multi-modal translation system improves translation quality and improves the quality of the captions of the images.

Copied to clipboard

Alham Fikri Aji, Kenneth Heafield

English-Myanmar Supervised and Unsupervised NMT: NICT’s Machine Translation Systems at WAT-2019 (D19-52)

Challenge:	Asynchronous stochastic gradient descent (SGD) converges poorly for Transformer models . synchronous SGD is faster at raw training speed since it avoids waiting for synchronization .
Approach:	They propose a method to restore convergence by summing several asynchronous updates instead of applying them immediately.
Outcome:	The proposed method achieves the same BLEU score 1.36 times faster than asynchronous SGD.

Copied to clipboard

Rui Wang, Haipeng Sun, Kehai Chen, Chenchen Ding, Masao Utiyama, Eiichiro Sumita

More Parameters? No Thanks! (2021.findings-acl)

Challenge:	NICT participated in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (My) - English task in both translation directions.
Approach:	They present the participation of the NICT in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (Burmese) - English task in both translation directions.
Outcome:	The proposed systems perform the third in English-to-Myanmar and the second in Myanmar-to English according to BLEU score.

Copied to clipboard

Zeeshan Khan, Kartheek Akella, Vinay Namboodiri, C V Jawahar

Green & Sustainable NLP

Challenge:	Using network pruning, we find that there are large redundancies in MNMT models.
Approach:	They propose a method to prune and retrain redundant parameters of an MNMT model to improve bilingual representations while retaining multilinguality.
Outcome:	The proposed method improves bilingual representations while retaining multilinguality.

Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)

Copied to clipboard

Hiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama, Hideki Tanaka, Eiichiro Sumita

Cross-lingual Supervision Improves Unsupervised Neural Machine Translation (2021.naacl-industry)

Challenge:	k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training.
Approach:	They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval.
Outcome:	The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja.

Copied to clipboard

Mingxuan Wang, Hongxiao Bai, Hai Zhao, Lei Li

Document-Level Neural Machine Translation Using BERT as Context Encoder (2020.aacl-srw)

Challenge:	Existing models that use only monolingual data have not been fully duplicated in the vast majority of language pairs, especially for zero-source languages.
Approach:	They propose to leverage the corpus from En-Fr and En-De to collectively train the translation from one language into many languages under one model.
Outcome:	The proposed model significantly improves translation quality with a big margin in the benchmark unsupervised translation tasks and achieves comparable performance to supervised NMT.

Copied to clipboard

Zhiyu Guo, Minh Le Nguyen

Converting the Point of View of Messages Spoken to Virtual Assistants (2020.findings-emnlp)

Challenge:	Large-scale pre-trained representations such as BERT have been widely used in many natural language understanding tasks.
Approach:	They propose to use BERT as a context encoder to achieve document-level contextual information, which is then integrated into both the encoder and decoder.
Outcome:	The proposed model outperforms strong document-level machine translation baselines on BLEU score and captures document- level context information to boost translation performance.

Copied to clipboard

Gunhee Lee, Vera Zu, Sai Srujana Buddi, Dennis Liang, Purva Kulkarni, Jack FitzGerald

Paraphrases as Foreign Languages in Multilingual Neural Machine Translation (P19-2)

Challenge:	Using a voice message, virtual assistants extract the message and send it to the user’s contact, rather than properly converting it to “I love you.”
Approach:	They propose to take a voice message from one user, convert it to “I love you” and deliver it to its target user.
Outcome:	The proposed system can take a voice message from one user, convert the point of view of the message, and then deliver the result to its target user.

Copied to clipboard

Zhong Zhou, Matthias Sperber, Alexander Waibel

Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models (P18-3)

Challenge:	Unlike previous studies that use paraphrases at the word/phrase level, we train on parallel paraphrase training on closely related languages.
Approach:	They train on parallel paraphrases in the style of multilingual Neural Machine Translation (NMT) they train on translations of the whole corpus that are consistent in structure as paraphrase versions at the corpus level.
Outcome:	The proposed training on paraphrases outperforms the baselines on two languages and improves lexical choice and entropy.

Copied to clipboard

Satoru Katsumata, Yukio Matsumura, Hayahide Yamagishi, Mamoru Komachi

Reference Free Domain Adaptation for Translation of Noisy Questions with Question Specific Rewards (2023.findings-emnlp)

Challenge:	Encoder-decoder models employ words that are frequently used in the training corpus but may still include noisy words.
Approach:	They propose a method for selecting more suitable words for learning encoders by utilizing co-occurrence information.
Outcome:	The proposed method outperforms the baseline method in Japanese-to-English translation and grammatical error correction tasks with an F-measure of 1.48 points higher.

Copied to clipboard

Baban Gain, Ramakrishna Appicharla, Soumya Chennabasavaraj, Nikesh Garera, Asif Ekbal, Muthusamy Chelliah

Query and Output: Generating Words by Querying Distributed Word Representations for Paraphrase Generation (N18-1)

Challenge:	Creating a synthetic parallel corpus from noisy data is also difficult due to its noisy nature.
Approach:	They propose a training methodology that fine-tunes the NMT system only using source-side data to balance adequacy and fluency.
Outcome:	The proposed method surpasses the MLE-based fine-tuning approach by achieving a 1.9 BLEU improvement.

Copied to clipboard

Shuming Ma, Xu Sun, Wei Li, Sujian Li, Wenjie Li, Xuancheng Ren

Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks (N18-4)

Challenge:	Existing models tend to memorize words instead of learning meaning of words . existing models tend not to model semantic information, resulting in incorrect sentences .
Approach:	They propose a novel model that generates words by querying distributed word representations . they evaluate model on two paraphrase-oriented tasks, namely text simplification and short abstractive summarization .
Outcome:	The proposed model outperforms the baseline model on two paraphrase-oriented tasks . it achieves state-of-the-art performance on these benchmark datasets .

Copied to clipboard

Vasu Jindal

Better Chinese Sentence Segmentation with Reinforcement Learning (2021.findings-acl)

Challenge:	Existing studies on image caption generation in English focus on Western languages, ignoring Semitic and Middle-Eastern languages like Arabic, Hebrew, Urdu and Persian.
Approach:	They propose to leverage the critical dependency of Arabic to generate Arabic captions using root-word based Recurrent Neural Network and Deep Neural networks.
Outcome:	The proposed model outperforms English-Arabic translated captions on a dataset from newspapers in the Middle East.

Copied to clipboard

Srivatsan Srinivasan, Chris Dyer

Sentiment Aware Neural Machine Translation (D19-52)

Challenge:	Chinese-English machine translation systems use ambiguous sentence boundaries, but English and Chinese use different orthographic conventions to designate sentence boundaries.
Approach:	They propose a segmentation policy that splits Chinese texts into segments that can be independently translated to maximise translation quality.
Outcome:	The proposed method improves the baseline BLEU score on the Chinese-English news translation task by +0.3 BLUE overall and the score on input segments that contain more than 60 words by +3 BL EU.

Copied to clipboard

Chenglei Si, Kui Wu, Ai Ti Aw, Min-Yen Kan

Monash University’s Submissions to the WNGT 2019 Document Translation Task (D19-56)

Challenge:	Sentiment ambiguous lexicons are used when context is absent in translations . most systems aim to produce one correct translation for a given source sentence .
Approach:	They propose a neural machine translation method that preserves sentiment in two sentiment scenarios and a method that embeds sentiment into a sentence.
Outcome:	The proposed method outperforms a baseline with sentiment-aware translations in both the BLEU score and translation accuracy.

Copied to clipboard

Sameen Maruf, Gholamreza Haffari

Choosing What to Mask: More Informed Masking for Multimodal Machine Translation (2023.acl-srw)

Challenge:	Despite the boom of work on document-level machine translation in the past two years, there has been a lack of the application of the proposed approaches to MT shared tasks.
Approach:	They propose to employ an established document-level neural machine translation model for the shared task of Rotowire document translation organised by the 3rd Workshop on Neural Generation and Translation (WNGT 2019).
Outcome:	The proposed model achieves a BLEU score of 39.83 for En-De and 45.06 for De-En translation directions on the Rotowire test set.

Copied to clipboard

Julia Sato, Helena Caseli, Lucia Specia

Multiscale Collaborative Deep Models for Neural Machine Translation (2020.acl-main)

Challenge:	Pre-trained language models have achieved remarkable results on several NLP tasks.
Approach:	They propose three new masking strategies for cross-lingual visual pre-training that focus on learning different linguistic patterns.
Outcome:	The proposed methods outperform the baseline model and achieve state-of-the-art accuracy on the Portuguese-English MMT task.

Copied to clipboard

Xiangpeng Wei, Heng Yu, Yue Hu, Yue Zhang, Rongxiang Weng, Weihua Luo

Simultaneous Speech-to-Text Translation Web Application for Estonian (2026.eacl-demo)

Challenge:	Neural machine translation models with deeper neural networks are difficult to train.
Approach:	They propose a MultiScale Collaborative framework to boost gradient back-propagation . they let each encoder block learn a fine-grained representation and enhance it .
Outcome:	The proposed framework outperforms baseline models on translation tasks with three translation directions and achieves a BLEU score of 30.56 on the English-to-German task.

Copied to clipboard

Bohdan Podziubanchuk, Aivo Olev, Jiaming Kong, Tanel Alumäe

Norm-Based Curriculum Learning for Neural Machine Translation (2020.acl-main)

Challenge:	a new open-source web application for simultaneous speech-to-text translation is developed for Estonian . the system translates live Estonian speech into English, Russian, and Ukrainian text, and also supports English-to Estonian translation.
Approach:	They propose a web application that combines streaming speech recognition with a simultaneous translation model.
Outcome:	The proposed system outperforms existing streaming speech recognition systems in Estonian-to-English translation.

Copied to clipboard

Xuebo Liu, Houtim Lai, Derek F. Wong, Lidia S. Chao

Lost in Machine Translation: A Method to Reduce Meaning Loss (N19-1)

Challenge:	Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).
Approach:	They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding.
Outcome:	The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).

Copied to clipboard

Reuben Cohn-Gordon, Noah Goodman

Semi-Autoregressive Neural Machine Translation (D18-1)

Challenge:	state-of-the-art translation systems often fail in preserving meaning . ambiguity between source and target languages can cause translation problems .
Approach:	They propose to use a pre-trained neural sequence-to-sequence model to define a less ambiguous translation system.
Outcome:	The proposed system preserves meaning in two languages without compromising translation quality.

Copied to clipboard

Chunqi Wang, Ji Zhang, Haiqing Chen

Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training (2022.findings-acl)

Challenge:	Existing approaches to neural machine translation are typically autoregressive but suffer from low parallelizability and thus slow at decoding long sequences.
Approach:	They propose a semi-autoregressive Transformer model for fast sequence generation that keeps the autoregressive property in global but relieves in local .
Outcome:	The proposed model achieves 5.58 speedup while maintaining 88% translation quality, significantly better than previous non-autoregressive methods.

Copied to clipboard

Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, Gholamreza Haffari

Green & Sustainable NLP

Challenge:	Recent advances in neural machine translation (NMT) research have found NMT models sensitive to distribution shift and adversarial examples.
Approach:	They propose a leave-one-domain-out training strategy that learns to combine domain-specific parameters to avoid information leaking.
Outcome:	The proposed method outperforms baselines on three language pairs on average.

Bag-of-Words as Target for Neural Machine Translation (P18-2)

Copied to clipboard

Shuming Ma, Xu Sun, Yizhong Wang, Junyang Lin

Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)

Challenge:	Existing neural machine translation models only use one correct sentence as the target, and the other correct sentences are punished as the incorrect ones.
Approach:	They propose an approach that uses both the sentences and the bag-of-words as targets in the training stage to encourage the model to generate the potentially correct sentences that are not appeared in the train set.
Outcome:	The proposed model outperforms baseline models on a Chinese-English translation dataset by the BLEU score of 4.55.

Copied to clipboard

Bei Li, Ziyang Wang, Hui Liu, Yufan Jiang, Quan Du, Tong Xiao, Huizhen Wang, Jingbo Zhu

AirConcierge: Generating Task-Oriented Dialogue via Efficient Large-Scale Knowledge Retrieval (2020.findings-emnlp)

Challenge:	Experimental results show that deep training is 1:4 faster than training from scratch.
Approach:	They propose a shallow-to-deep training method that learns deep models by stacking shallow models.
Outcome:	The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks.

Copied to clipboard

Chieh-Yang Chen, Pei-Hsin Wang, Shih-Chieh Chang, Da-Cheng Juan, Wei Wei, Jia-Yu Pan

Context-Aware Monolingual Repair for Neural Machine Translation (D19-1)

Challenge:	Existing neural task-oriented dialogue systems cannot be encoded by memory networks, such as memory networks.
Approach:	They propose an end-to-end trainable text-to SQL guided framework to learn a neural agent that interacts with KBs using the generated SQL queries.
Outcome:	The proposed method significantly improves on the AirDialogue dataset, which contains the conversations of customers booking flight tickets from the agent.

Copied to clipboard

Elena Voita, Rico Sennrich, Ivan Titov

Using Word Vectors to Improve Word Alignments for Low Resource Machine Translation (N18-2)

Challenge:	et al., 2018) show that human raters prefer corrected translations over the baseline ones.
Approach:	They propose a monolingual model to correct inconsistencies between sentences . they use monolingual document-level data to train the model .
Outcome:	The proposed model improves translations of contextual phenomena in English-Russian translation task.

Copied to clipboard

Nima Pourdamghani, Marjan Ghazvininejad, Kevin Knight

PoMo: Generating Entity-Specific Post-Modifiers in Context (N19-1)

Challenge:	Using word similarities, we improve word alignments in low resource settings . word alignment is essential for statistical machine translation (MT)
Approach:	They propose a method for improving word alignments using word similarities using word vectors trained on monolingual data.
Outcome:	The proposed method improves word alignments in low-resource settings by improving alignments of infrequent tokens.

Copied to clipboard

Jun Seok Kang, Robert Logan, Zewei Chu, Yang Chen, Dheeru Dua, Kevin Gimpel, Sameer Singh, Niranjan Balasubramanian

Challenge:	Using crowdsourcing, we show that contextual relevance is necessary for accurate post-modifier generation.
Approach:	They introduce entity post-modifier generation as an instance of a collaborative writing task . they build a post- modifier dataset from news articles that provides contextually relevant information about the target entity.
Outcome:	The proposed system can generate a post-modifier phrase that provides contextually relevant information about the target entity.

Few-shot learning through contextual data augmentation (2021.eacl-main)

Copied to clipboard

Farid Arthaud, Rachel Bawden, Alexandra Birch

Low-Resource NLP

Challenge:	Various strategies have been explored to learn from a journalist's post-edits . state-of-the-art APE systems require large numbers of post- edits for training .
Approach:	They propose to teach a pre-trained machine translation model to translate previously unseen words accurately . they extend a data augmentation approach to create training examples with similar contexts .
Outcome:	The proposed model improves accuracy on the scale of one to five examples with only 1 to 5 examples.

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach (P18-1)

Copied to clipboard

Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, Wenjie Li

Multilingual and Multimodal Learning for Brazilian Portuguese (2022.lrec-1)

Challenge:	Existing studies for sentiment-to-sentiment "translation" only change the underlying sentiment and fail to keep the semantic content.
Approach:	They propose a cycled reinforcement learning method that combines neutralization module and emotionalization module.
Outcome:	The proposed method outperforms state-of-the-art systems on Yelp and Amazon review datasets.

Copied to clipboard

Júlia Sato, Helena Caseli, Lucia Specia

Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data (2021.acl-short)

Challenge:	Existing models that learn multimodal and multilingual representations perform better in many natural language tasks.
Approach:	They use a multimodal and multilingual corpus to test its generalization ability for other languages . they achieve a BLEU score of 51.8 and a METEOR score of 78.0 on the test set .
Outcome:	The proposed model outperforms the existing model on a Portuguese-English multimodal translation task.

Copied to clipboard

Sajad Norouzi, Keyi Tang, Yanshuai Cao

Programming Languages in NLP

Challenge:	a generic transformer-based model can achieve competitive performance with minimal code-generation-specific inductive bias design.
Approach:	They investigate whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design.
Outcome:	The proposed model achieves 81.03% exact match accuracy on Django and 32.57 BLEU score on CoNaLa.

Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (P18-2)

Copied to clipboard

Poorya Zaremoodi, Wray Buntine, Gholamreza Haffari

Low-Resource NLP

Challenge:	Neural Machine Translation (NMT) requires large amounts of bilingual data to learn a translation model with reasonable quality.
Approach:	They propose to extend recurrent units with multiple "blocks" along with a trainable "routing network" this allows for adaptive collaboration by dynamic sharing of blocks conditioned on the task at hand, input, and model state.
Outcome:	Empirical evaluations of two low-resource translation tasks show +1 BLEU score improvements compared to strong baselines.

Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help? (2021.findings-emnlp)

Copied to clipboard

Fahimeh Saleh, Wray Buntine, Gholamreza Haffari, Lan Du

BERTTune: Fine-Tuning Neural Machine Translation with BERTScore (2021.acl-short)

Challenge:	Multilingual Neural Machine Translation (MNMT) trains a single model that supports translation between multiple languages . transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer.
Approach:	They propose a hierarchical knowledge distillation approach to train multilingual models . they use typological features and phylogeny to overcome negative transfer issue .
Outcome:	The proposed approach avoids negative transfer effect by capitalising on language groups generated according to typological features and phylogeny of languages.

Copied to clipboard

Inigo Jauregi Unanue, Jacob Parnell, Massimo Piccardi

Forest-Based Neural Machine Translation (P18-1)

Challenge:	Neural machine translation models are biased toward limited translation references . BERTScore is a scoring function based on contextual embeddings that overcomes the limitations of n-gram-based metrics.
Approach:	They propose to fine-tune models with a new evaluation metric based on contextual embeddings to overcome the limitations of n-gram-based metrics.
Outcome:	The proposed training objective improves translations that are different from the translations but close in the contextual embedding space.

Copied to clipboard

Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, Eiichiro Sumita

Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations (N19-1)

Challenge:	Compared with string-to-string systems, tree-based NMT methods use more syntactic information and can incorporate prior knowledge.
Approach:	They propose a tree-based neural machine translation method that translates a linearized packed forest under a simple sequence-to-sequence framework.
Outcome:	The proposed method outperforms tree-based approaches in the BLEU score of the proposed model.

Copied to clipboard

Meishan Zhang, Zhenghua Li, Guohong Fu, Min Zhang

Handling Homographs in Neural Machine Translation (N18-1)

Challenge:	Syntax integration has been demonstrated highly effective in neural machine translation (NMT).
Approach:	They propose a method to integrate source-side syntax implicitly for neural machine translation . they use hidden representations of a well-trained end-to-end dependency parser to concatenate them with ordinary word embeddings to enhance basic NMT models.
Outcome:	The proposed method outperforms existing methods on two translation tasks . it can be easily integrated into the widely-used sequence-to-sequence (Seq2Sequen) framework .

Copied to clipboard

Frederick Liu, Han Lu, Graham Neubig

Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue (D19-1)

Challenge:	Existing methods for MT have problems with translating homographs, as it is difficult to select the correct translation based on the context.
Approach:	They propose to model the context of the input word with context-aware word embeddings that help to differentiate the word sense before feeding it into the encoder.
Outcome:	The proposed models improve translation accuracy and BLEU score on three language pairs.

Copied to clipboard

Chenguang Zhu, Michael Zeng, Xuedong Huang

Encouraging Neural Machine Translation to Satisfy Terminology Constraints (2021.findings-acl)

Challenge:	Existing methods to generate natural language for task-oriented dialogues lack naturalness and variation in language.
Approach:	They propose a multi-task learning framework for natural language generation that explicitly targets for naturalness in generated responses via an unconditioned language model.
Outcome:	The proposed framework outperforms existing models across multiple datasets in the study of natural language generation.

Copied to clipboard

Melissa Ailem, Jingshu Liu, Raheel Qader

Distilling Knowledge for Search-based Structured Prediction (P18-1)

Challenge:	a new approach to encourage neural machine translation to satisfy lexical constraints is proposed . a BLEU score and percentage of generated constraint terms are improved by the proposed method .
Approach:	They propose a method that encourages neural machine translation to satisfy lexical constraints at training step . they use a simplified augmentation strategy without source factors and constraint token masking to make it easier to learn the copy behavior .
Outcome:	The proposed method improves on baselines in terms of BLEU score and percentage of generated constraint terms.

Copied to clipboard

Yijia Liu, Wanxiang Che, Huaipeng Zhao, Bing Qin, Ting Liu

Rethinking Sentiment Style Transfer (2021.findings-emnlp)

Challenge:	Existing studies have focused on the performance of structured prediction models, but they are often limited by the ambiguities of the reference policy.
Approach:	They propose to distill an ensemble of multiple models trained with different initializations into a single model and use it to explore the search space.
Outcome:	The proposed model outperforms the greedy models on two typical search-based structured prediction tasks and achieves 1.32 in LAS and 2.65 in BLEU over strong baselines.

Copied to clipboard

Ping Yu, Yang Zhao, Chunyuan Li, Changyou Chen

Sentiment Analysis

Challenge:	Existing evaluation methods for text style transfer are unsatisfactory.
Approach:	They propose to use a graph-based method to extract attribute content from sentences . they propose an efficient regularization to leverage attribute-dependent content as guiding signals.
Outcome:	The proposed method is based on a YELP and IMDB dataset and it is able to detect errors in the human evaluation.

Learning to Collaborate for Question Answering and Asking (N18-1)

Copied to clipboard

Duyu Tang, Nan Duan, Zhao Yan, Zhirui Zhang, Yibo Sun, Shujie Liu, Yuanhua Lv, Ming Zhou

RedApt: An Adaptor for wav2vec 2 EncodingFaster and Smaller Speech Translation without Quality Compromise (2022.findings-emnlp)

Challenge:	Question answering (QA) and question generation (QG) are closely related tasks.
Approach:	They propose a training algorithm that generalizes both Generative Adversarial Network and Generating Domain-Adaptive Nets under the question answering scenario.
Outcome:	The proposed training algorithm generalizes both Generative Adversarial Network (GAN) and Generating Domain-Adaptive Nets (GDAN) under the question answering scenario.

Copied to clipboard

Jinming Zhao, Hao Yang, Gholamreza Haffari, Ehsan Shareghi

Speech & Audio in NLP

Challenge:	Pre-trained speech Transformers in speech translation systems have facilitated state-of-the-art (SotA) results, but their computational cost is high.
Approach:	They propose a Reducer Adaptor block that could be seamlessly integrated within any Transformer-based speech encoding architecture.
Outcome:	The proposed Reducer Adaptor block outperforms the existing SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C.

Exploiting Multilingualism through Multistage Fine-Tuning for Low-Resource Neural Machine Translation (D19-1)

Copied to clipboard

Raj Dabre, Atsushi Fujita, Chenhui Chu

Generating Synthetic Speech from SpokenVocab for Speech Translation (2023.findings-eacl)

Challenge:	Using multi-parallel corpora for transfer learning is a useful technique for low-resource NMT.
Approach:	They compare multi-parallel corpora for transfer learning in a low-resource setting . their results show that multi-paralleled corpors are extremely useful .
Outcome:	The proposed model can give 3–9 BLEU score gains over a one-to-one model.

Copied to clipboard

Jinming Zhao, Gholamreza Haffari, Ehsan Shareghi

Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation (2020.findings-emnlp)

Challenge:	End-to-end speech-totext translation (ST) models require large amounts of data to train, but their size is considerably smaller than text-based MT data.
Approach:	They propose a method to convert MT data to ST data via text-to-speech systems.
Outcome:	The proposed method improves translation quality by an average of 1.83 BLEU score while performing equally well as TTS-generated speech in improving translation quality.

Copied to clipboard

Baohao Liao, Yingbo Gao, Hermann Ney

Fixed That for You: Generating Contrastive Claims with Semantic Edits (N19-1)

Challenge:	Neural machine translation (NMT) has achieved significant progress over recent years.
Approach:	They extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level.
Outcome:	The proposed method improves on the IWSLT’14 German-English task and also on the WMT’14 English-German task.

Copied to clipboard

Christopher Hidey, Kathy McKeown

Challenge:	Understanding contrastive opinions is a key component of argument generation.
Approach:	They create a corpus of Reddit comment pairs and train neural models to edit the original claim and produce a new claim with a different view.
Outcome:	The proposed model improves on a sequence-to-sequence baseline and compared to a human evaluation for fluency, coherence, and contrast.

Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, Ke Xu

Data-Efficient Goal-Oriented Conversation with Dialogue Knowledge Transfer Networks (D19-1)

Challenge:	DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting .
Approach:	They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting .
Outcome:	The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks.

Copied to clipboard

Igor Shalyminov, Sungjin Lee, Arash Eshghi, Oliver Lemon

Dialogue Systems & Conversational Agents

Challenge:	Goal-oriented dialogue systems are now being widely adopted in industry where it is of key importance to maintain a rapid prototyping cycle for new products and domains.
Approach:	They propose a data-driven approach to goal-oriented dialogue generation which only uses a few example dialogues, none of which has to be annotated.
Outcome:	The proposed approach significantly improves upon baseline models and over the previous state-of-the-art model, ZSDG.

Learning to Stop in Structured Prediction for Neural Machine Translation (N19-1)

Copied to clipboard

Mingbo Ma, Renjie Zheng, Liang Huang

Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation (P19-1)

Challenge:	Beam search optimization solves many problems in neural machine translation, but lacks principled stopping criteria and does not learn how to stop during training.
Approach:	They propose a ranking method which enables an optimal beam search stop-ping criteria and a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training.
Outcome:	Experiments on synthetic and real languages show that the proposed methods improve translation quality and length.

Copied to clipboard

Shuming Ma, Pengcheng Yang, Tianyu Liu, Peng Li, Jie Zhou, Xu Sun

Direct Segmentation Models for Streaming Speech Translation (2020.emnlp-main)

Challenge:	Existing methods for table-to-text generation use encoder-decoder framework, but lack of large parallel data is a problem for many domains.
Approach:	They propose a model to separate table-to-text generation into two stages: key fact prediction and surface realization.
Outcome:	The proposed model achieves 27.34 BLEU score with only 1,000 parallel data, while the baseline model only achieves 9.71 BLUE score.

Copied to clipboard

Javier Iranzo-Sánchez, Adrià Giménez Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal, Jorge Civera Saiz, Alfons Juan

Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation (2020.emnlp-main)

Challenge:	Existing approaches to stream ST combine advances in ASR and MT to achieve high quality translations without compromising the speed of the system.
Approach:	They propose to concatenate an Automatic Speech Recognition system followed by a Machine Translation system.
Outcome:	The proposed models improve on the Europarl-ST dataset on the BLEU score.

Copied to clipboard

Tahmid Hasan, Abhik Bhattacharjee, Kazi Samin, Masum Hasan, Madhusudan Basak, M. Sohel Rahman, Rifat Shahriyar

Adaptive Feature Selection for End-to-End Speech Translation (2020.findings-emnlp)

Challenge:	despite being the seventh most widely spoken language, Bengali has received little attention in machine translation due to being low in resources.
Approach:	They propose a customized sentence segmenter for Bengali and two new methods for parallel corpus creation on low-resource setups.
Outcome:	The proposed method improves Bengali-English parallel corpus by 9 BLEU over previous approaches . the results will pave the way for future research on Bengali and other low-resource languages .

Copied to clipboard

Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Neural Text Generation from Rich Semantic Representations (N19-1)

Challenge:	End-to-end speech translation (E2E) models that directly maps audio to a foreign text are not efficient.
Approach:	They propose adaptive feature selection (AFS) for encoder-decoder based E2E ST.
Outcome:	The proposed model outperforms the existing model on LibriSpeech En-Fr and MuST-C with a BLEU score of 18.56.

Copied to clipboard

Valerie Hajdik, Jan Buys, Michael Wayne Goodman, Emily M. Bender

Representation Learning

Challenge:	2 is a neural model that maps a linearization of Dependency MRS to text . 1 is based on a BLEU score of 66.11 when trained on gold data .
Approach:	They propose to use Minimal Recursion Semantics to generate high-quality text from structured representations.
Outcome:	The proposed model achieves a BLEU score of 77.17 on the full test set and 83.37 on the subset of test data most closely matching the silver data domain.

DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models (2023.acl-long)

Copied to clipboard

Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuanjing Huang, Xipeng Qiu

Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition (2020.findings-emnlp)

Challenge:	Existing generative masked language models have a shared training objective, i.e., denoising.
Approach:	They propose a noise schedule for the forward diffusion process that controls the degree of noise added at each step based on the information of each token.
Outcome:	The proposed model improves on existing models in terms of perplexity and BLEU score.

Copied to clipboard

Vasileios Lioutas, Ahmad Rashid, Krtin Kumar, Md. Akmal Haidar, Mehdi Rezagholizadeh

PharmMT: A Neural Machine Translation Approach to Simplify Prescription Directions (2020.findings-emnlp)

Challenge:	Word-embeddings are vital components of natural language processing (NLP) but they consume a lot of memory which poses a challenge for edge deployment.
Approach:	They propose an embedding compression method based on matrix decomposition and knowledge distillation that initializes weights of pre-trained word-embeddings and fine-tunes end-to-end.
Outcome:	The proposed method has higher BLEU score on translation and lower perplexity on language modeling compared to complex, difficult to tune methods.

Copied to clipboard

Jiazhao Li, Corey Lester, Xinyan Zhao, Yuting Ding, Yun Jiang, V.G.Vinod Vydiswaran

Handling Rare Word Problem using Synthetic Training Data for Sinhala and Tamil Neural Machine Translation (L18-1)

Challenge:	a novel machine translation-based approach to simplify prescription directions is proposed . the language used by physicians and health professionals includes medical jargon and implicit directives .
Approach:	They propose a machine translation-based approach to automatically and reliably simplify prescription directions into patient-friendly language.
Outcome:	The proposed system achieves a BLEU score of 60.27 over 530K prescriptions from a large mail-order pharmacy.

Copied to clipboard

Pasindu Tennage, Prabath Sandaruwan, Malith Thilakarathne, Achini Herath, Surangika Ranathunga

Language Tags Matter for Zero-Shot Neural Machine Translation (2021.findings-acl)

Challenge:	Lack of parallel training data influences rare word problem in Neural Machine Translation systems, especially for underresourced languages.
Approach:	They propose to use Parts of Speech tagging and morphological analysis as syntactic features to prune generated synthetic sentence pairs that do not adhere to language syntax.
Outcome:	The proposed methods show that they can prune sentences that do not adhere to language syntax over Sinhala to Tamil and Tamil to Sinhalak translation systems.

Copied to clipboard

Liwei Wu, Shanbo Cheng, Mingxuan Wang, Lei Li

Understanding Jargon: Combining Extraction and Generation for Definition Modeling (2022.emnlp-main)

Challenge:	Existing studies on multilingual machine translation have ignored the importance of LTs.
Approach:	They propose to use language tag (LT) strategies to indicate translation directions in MNMT to enhance consistency and alleviate off-target issues in zero-shot directions.
Outcome:	The proposed model could translate between unsupervised languages and achieve a +8 BLEU score difference over other LT strategies in translation tasks.

Copied to clipboard

Jie Huang, Hanyin Shao, Kevin Chen-Chuan Chang, Jinjun Xiong, Wen-mei Hwu

Information Extraction & Text Mining

Challenge:	Existing methods for generating definitions of words/phrases perform poorly on jargon.
Approach:	They propose to combine extraction and generation for jargon definition modeling by extracting definitional information from the Web and incorporating extracted definitional data.
Outcome:	The proposed method outperforms state-of-the-art models significantly on jargon definitions.

Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection (2021.emnlp-main)

Copied to clipboard

Thuy-Trang Vu, Xuanli He, Dinh Phung, Gholamreza Haffari

Look Harder: A Neural Machine Translation Model with Hard Attention (P19-1)

Challenge:	Existing work on unsupervised domain adaptation of neural machine translation assumes access to monolingual text in either the source or target language in the new domain.
Approach:	They propose a method to extract in-domain sentences from a large generic monolingual corpus from 'missing' text.
Outcome:	The proposed method outperforms baselines up to +1.5 BLEU score on five diverse domains in three language pairs and a real-world translation scenario.

Copied to clipboard

Sathish Reddy Indurthi, Insoo Chung, Sangha Kim

Disfluency Correction using Unsupervised and Semi-supervised Learning (2021.eacl-main)

Challenge:	Soft-attention based Neural Machine Translation models attend all the words in the source sequence for each target token, which makes them ineffective for long sequence translation.
Approach:	They propose a hard-attention based NMT model which selects a subset of source tokens for each target token to effectively handle long sequence translation.
Outcome:	The proposed model performs better on long sequences and achieves significant improvement on English-German and English-French translation tasks compared to soft-attention based models.

Copied to clipboard

Nikhil Saini, Drumil Trivedi, Shreya Khare, Tejas Dhamecha, Preethi Jyothi, Samarth Bharadwaj, Pushpak Bhattacharyya

Low-Resource NLP

Challenge:	Disfluencies in conversational speech can affect performance of downstream NLP tasks.
Approach:	They propose a disfluency correction model that converts disfluent to fluent text . they use unsupervised encoder-decoder models to generate semi-supervised models .
Outcome:	The proposed model achieves a BLEU score of 79.39 on the Switchboard corpus test set and 85.28 with semi-supervision.

Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation (2020.coling-main)

Copied to clipboard

Fahimeh Saleh, Wray Buntine, Gholamreza Haffari

Responsible & Trustworthy NLP

Challenge:	Existing approaches to train high-quality NMT models in bilingually low-resource scenarios are limited by the scarcity of parallel sentence-pairs.
Approach:	They propose to distill the knowledge of teacher models to a single student model by using knowledge distillation.
Outcome:	The proposed approach achieves up to +0.9 BLEU score improvements compared to strong baselines.

“Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer (D19-1)

Copied to clipboard

Akhilesh Sudhakar, Bhargav Upadhyay, Arjun Maheswaran

Probing Multi-modal Machine Translation with Pre-trained Language Model (2021.findings-acl)

Challenge:	Text style transfer is the task of transferring the style of text having certain stylistic attributes, while preserving non-stylistic or content information.
Approach:	They propose a new approach to rewriting sentences to a target style in the absence of parallel style corpora by exploiting the Transformer.
Outcome:	The proposed method outperforms state-of-the-art systems across 5 datasets on sentiment, gender and political slant transfer.

Copied to clipboard

Kong Yawei, Kai Fan

Multimodality

Challenge:	Multi-modal machine translation (MMT) aimed at using images to help disambiguate the target during translation but recent studies showed that visual features are either negligible or incremental.
Approach:	They propose to incorporate a visual language model on the source side to improve multi-modal translation quality significantly.
Outcome:	The proposed model improves the translation quality significantly on the multi-modal dataset.

Document-Level Neural Machine Translation with Hierarchical Attention Networks (D18-1)

Copied to clipboard

Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, James Henderson

Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning (2024.findings-emnlp)

Challenge:	Neural machine translation (NMT) can be improved by including document-level contextual information.
Approach:	They propose a hierarchical attention model that captures document-level contextual information and conditioning on the NMT model’s own hidden states.
Outcome:	The proposed model improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods and that both the encoder and decoder benefit from context in complementary ways.

Copied to clipboard

Tianxiang Hu, Pei Zhang, Baosong Yang, Jun Xie, Derek Wong, Rui Wang

Automatic Post-Editing of Machine Translation: A Neural Programmer-Interpreter Approach (D18-1)

Challenge:	Achieving consistent high-quality machine translation across diverse domains remains a challenge due to limited and imbalanced parallel training data available in various domains.
Approach:	They propose a domain Chain of Thought technique that uses the multi-domain intelligence of LLMs to improve translation performance.
Outcome:	The proposed method achieves significant improvements in translation accuracy and domain robustness over traditional fine-tuning on a small dataset of four domains.

Copied to clipboard

Thuy-Trang Vu, Gholamreza Haffari

SDS-200: A Swiss German Speech to Standard German Text Corpus (2022.lrec-1)

Challenge:	Existing approaches to inducing APE have suffered from over-correction, where the APE system tends to keep the machine translated text without any modification.
Approach:	They propose a neural programmer-interpreter approach to automated post-editing (APE) that mimics human perform post- editing using discrete edit operations . their model outperforms previous neural models for inducing PE programs on the WMT17 APE task for German-English up to +1 BLEU score and -0.7 TER scores.
Outcome:	The proposed model outperforms previous neural models for inducing PE programs on the WMT17 APE task for German-English up to +1 BLEU score and -0.7 TER scores.

Copied to clipboard

Michel Plüss, Manuela Hürlimann, Marc Cuny, Alla Stöckli, Nikolaos Kapotis, Julia Hartmann, Malgorzata Anna Ulasik, Christian Scheller, Yanick Schraner, Amit Jain, Jan Deriu, Mark Cieliebak, Manfred Vogel

Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models (2021.acl-long)

Challenge:	Using a web recording tool, participants were asked to translate their Swiss German text to their own dialect before recording it.
Approach:	They present a corpus of Swiss German dialectal speech with Standard German text translations . the dataset allows for training speech translation, dialect recognition, and speech synthesis systems .
Outcome:	The dataset allows for training speech translation, dialect recognition, and speech synthesis systems.

Copied to clipboard

Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum

Improving Machine Translation with Phrase Pair Injection and Corpus Filtering (2022.emnlp-main)

Challenge:	Autoregressive neural machine translation (NMT) uses a tractable likelihood computation and efficient sampling.
Approach:	They propose to use an energy-based model to mimic the behavior of the task measure and use it to train an energy based re-ranking algorithm.
Outcome:	The proposed model improves on the samples drawn from the NMT with a higher BLEU score than the experimental model and the energy-based re-ranking algorithm.

Copied to clipboard

Akshay Batheja, Pushpak Bhattacharyya

A Myanmar (Burmese)-English Named Entity Transliteration Dictionary (2020.lrec-1)

Challenge:	In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation systems.
Approach:	They propose to combine Phrase Pair Injection and Corpus Filtering to boost performance of Neural Machine Translation systems.
Outcome:	The proposed method improves machine translation models on low-resource language pairs . BLEU score improves over models trained with whole pseudo-parallel corpus augmented with parallel corpus.

Copied to clipboard

Aye Myat Mon, Chenchen Ding, Hour Kaing, Khin Mar Soe, Masao Utiyama, Eiichiro Sumita

Challenge:	Currently, there are no data available for the transcription of borrowed English words in Myanmar . lack of resources is a problem for many understudied languages .
Approach:	They construct a dictionary of Myanmar-English transliteration instances using a CC BY-NC-SA license.
Outcome:	The proposed model outperforms the statistical model significantly on the character level.

Jam or Cream First? Modeling Ambiguity in Neural Machine Translation with SCONES (2022.naacl-main)

Copied to clipboard

Felix Stahlberg, Shankar Kumar

Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems (2020.coling-main)

Challenge:	Existing neural machine translation models learn the probability P (y\|x) of the target sentence given the source sentence x.
Approach:	They propose to replace softmax activation with a multi-label classification layer that can model ambiguity more effectively.
Outcome:	The proposed multi-label classification layer can model ambiguity more effectively . it yields consistent BLEU score gains across six translation directions .

Copied to clipboard

Vitou Phy, Yang Zhao, Akiko Aizawa

Affect-Driven Dialog Generation (N19-1)

Challenge:	Existing evaluation metrics are not designed to cope with this flexibility.
Approach:	They propose to group the qualities into three groups to obtain a single metric called USL-H.
Outcome:	The proposed metric achieves good correlations with human judgment and maintains its configurability towards different aspects and metrics.

Copied to clipboard

Pierre Colombo, Wojciech Witon, Ashutosh Modi, James Kennedy, Mubbasir Kapadia

Dialogue Systems & Conversational Agents

Challenge:	Existing systems for end-to-end dialog generation focus on response quality without explicit control over affective content of the responses.
Approach:	They propose an affect-driven dialog system which generates emotional responses using a continuous representation of emotions.
Outcome:	The proposed system outperforms existing systems in terms of BLEU score and response diversity, and qualitative measures.

Leveraging Discourse Rewards for Document-Level Neural Machine Translation (2020.coling-main)

Copied to clipboard

Inigo Jauregi Unanue, Nazanin Esmaili, Gholamreza Haffari, Massimo Piccardi

Emergent Communication Pretraining for Few-Shot Machine Translation (2020.coling-main)

Challenge:	Document-level machine translation models are often not trained to explicitly ensure discourse quality.
Approach:	They propose a method that explicitly optimizes lexical cohesion and coherence metrics by using a reinforcement learning objective.
Outcome:	The proposed approach improves document translations over four different languages and three translation domains while maintaining faithfulness to the reference translation.

Copied to clipboard

Yaoyiran Li, Edoardo Maria Ponti, Ivan Vulić, Anna Korhonen

SimplifyUR: Unsupervised Lexical Text Simplification for Urdu (2020.lrec-1)

Challenge:	state-of-the-art models that rely on multilingual pretrained encoders achieve sample efficiency in downstream applications, but lack abundant amounts of unlabelled text.
Approach:	They propose a method to pretrain neural networks via emergent communication from referential games by grounding communication on images as a crude approximation of real-world environments.
Outcome:	The proposed method significantly improves machine translation in few-shot settings and provides an evaluation protocol to probe the properties of emergent languages ex vitro.

Copied to clipboard

Namoos Hayat Qasmi, Haris Bin Zia, Awais Athar, Agha Ali Raza

Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity (D18-1)

Challenge:	Existing methods for text simplification for Urdu rely on manual lexicons and simplified corpora, but are not applicable to the language.
Approach:	They propose an unsupervised method for automatic text simplification for Urdu using word embeddings and morphological features.
Outcome:	The proposed method achieves BLEU score of 80.15 and SARI score of 42.02 on simple text generated on simplified corpora and human evaluations for correctness, grammaticality, meaning-preservation and simplicity.

Copied to clipboard

Xinnuo Xu, Ondřej Dušek, Ioannis Konstas, Verena Rieser

Dialogue Systems & Conversational Agents

Challenge:	Existing encoder-decoder models for open domain dialogue generate generic, uninformative, and non-coherent responses.
Approach:	They propose to introduce a measure of coherence as the GloVe embedding similarity between dialogue context and generated response to improve output diversity.
Outcome:	The proposed model improves on the OpenSubtitles corpus in terms of BLEU score and diversity metrics.

MMQA: A Multi-domain Multi-lingual Question-Answering Framework for English and Hindi (L18-1)

Copied to clipboard

Deepak Gupta, Surabhi Kumari, Asif Ekbal, Pushpak Bhattacharyya

Question Answering

Challenge:	Existing work on multi-domain, multi-lingual question answering is limited to the same language.
Approach:	They curate 500 articles in six different domains from the web and create question-answer pairs . they develop a deep learning based model for classifying an input question into coarse and finer categories .
Outcome:	The proposed model accuracies 90.12% and 80.30% for coarse and finer classes . the proposed model is the first attempt to create multi-domain, multi-lingual question answering evaluation involving English and Hindi.

Neural Machine Translation for Low-Resourced Indian Languages (2020.lrec-1)

Copied to clipboard

Himanshu Choudhary, Shivansh Rao, Rajesh Rohilla

Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020.emnlp-main)

Challenge:	Neural machine translation (NMT) is an effective way to convert text to a different language without human involvement.
Approach:	They propose to use multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient machine translation system.
Outcome:	The proposed system outperforms Google translator and the existing translators on two of the most morphological rich Indian languages.

Copied to clipboard

Eric Wallace, Mitchell Stern, Dawn Song

Robustness in NLP

Challenge:	Using simulated experiments, we demonstrate that MT systems can be stolen even when imitation models have different input data or architectures than their target models.
Approach:	They propose a defense that modifies translation outputs to misdirect optimization of imitation models.
Outcome:	The proposed defense degrades the adversary’s BLEU score and attack success rate at some cost in the defender’s performance and inference speed.

Iterative Constrained Back-Translation for Unsupervised Domain Adaptation of Machine Translation (2022.coling-1)

Copied to clipboard

Hongxiao Zhang, Hui Huang, Jiale Gao, Yufeng Chen, Jinan Xu, Jian Liu

Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation (2022.coling-1)

Challenge:	Existing back-translation methods focus on in-domain lexical knowledge, which may lead to poor translation of unseen in- domain words.
Approach:	They propose an iterative constrained back-translation method to incorporate in-domain lexical knowledge into synthetic parallel data from BT.
Outcome:	The proposed method improves the BLEU score by up to 3.08 on four domains.

Copied to clipboard

Qiang Wang, Rongxiang Weng, Ming Chen

ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms (2022.findings-emnlp)

Challenge:	Existing methods to integrate external corpus are sparse in practical applications, and noises in low similarity retrieval could lead to severe performance degradation.
Approach:	They propose a method to integrate external corpus into k-nearest neighbor machine translation (kNNMT) instead of storing discrete word sequence, kNN-MT uses a pre-trained NMT model to force decoding the external corpi.
Outcome:	The proposed approach improves retrieval accuracy and BLEU score on five domains compared to vanilla kNNMT.

Copied to clipboard

Jinxin Liu, Jiaxin Shi, Ji Qi, Lei Hou, Juanzi Li, Qi Tian

A Case Against Implicit Standards: Homophone Normalization in Machine Translation for Languages that use the Ge’ez Script. (2025.emnlp-main)

Challenge:	Existing unsupervised methods for paraphrase generation are weak in semantic equivalence or expression diversity.
Approach:	They propose a framework for unsupervised paraphrase generation that employs multi-aspect equivalence constraints and multi-granularity diversifying mechanisms to achieve good semantic equvalence and expressive diversity.
Outcome:	The proposed framework achieves 9.1% and 3.3% absolute gains over previous SOTA on Quora and MSCOCO and can improve to 18.0% and 4.6% on GLUE.

Copied to clipboard

Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Henok Biadglign Ademtew, Hizkiel Mitiku Alemayehu, Negasi Haile Abadi, Tadesse Destaw Belay, Seid Muhie Yimam

Semantic Captioning: Benchmark Dataset and Graph-Aware Few-Shot In-Context Learning for SQL2Text (2025.coling-main)

Challenge:	Homophone normalization is a pre-processing step used in Amharic natural language processing (NLP) but it also results in models that are unable to process different forms of writing in a single language.
Approach:	They propose a method where normalization is applied to model predictions instead of training data and a scheme where normalized data is preserved in training.
Outcome:	The proposed model achieves an increase in BLEU score of up to 1.03 while preserving language features in training.

Copied to clipboard

Ali Al Lawati, Jason Lucas, Prasenjit Mitra

Dodo: Dynamic Contextual Compression for Decoder-only LMs (2024.acl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable performance in various NLP tasks, including semantic parsing, which translates natural language into formal code representations.
Approach:	They propose a semantic captioning task to repurpose semantic parsing datasets for semantic captions.
Outcome:	The proposed model outperforms random selection and other methods by 39% on BLEU score.

Copied to clipboard

Guanghui Qin, Corby Rosset, Ethan Chau, Nikhil Rao, Benjamin Van Durme

It Is Not As Good As You Think! Evaluating Simultaneous Machine Translation on Interpretation Data (2021.emnlp-main)

Challenge:	Existing approaches to NLP are sparsifying attention patterns or approximating the attention computation with kernel methods.
Approach:	They propose a method for dynamic contextual compression for decoder-only LMs.
Outcome:	The proposed method reduces the cost of self-attention to a fraction of typical time and space.

Copied to clipboard

Jinming Zhao, Philip Arthur, Gholamreza Haffari, Trevor Cohn, Ehsan Shareghi

Learning Programmatic Idioms for Scalable Semantic Parsing (D19-1)

Challenge:	Existing siMT systems are trained and evaluated on offline translations . however, evaluation gap remains notable, calling for constructing large-scale interpretation corpora .
Approach:	They propose a translation-to-interpretation transfer method which converts offline translations into interpretation-style data.
Outcome:	The proposed interpretation test set shows that SiMT models improve on translation vs interpretation data.

Copied to clipboard

Srinivasan Iyer, Alvin Cheung, Luke Zettlemoyer

Multimodality

Challenge:	In state-of-the-art semantic parsers map natural language instructions to source code . idioms improve the accuracy of semantic parses, allowing for faster decoding .
Approach:	They propose an iterative method to extract code idioms from large source code corpora . they use most-frequent subtrees of their syntax trees to train semantic parsers to apply them .
Outcome:	The proposed method improves the state-of-the-art semantic parsers' accuracy and training time by more than 50%.

AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation (2023.findings-acl)

Copied to clipboard

Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu, Young Jin Kim, Muhammad Abdul-Mageed, Laks Lakshmanan, V.S., Ahmed Hassan Awadallah, Sebastien Bubeck, Jianfeng Gao

Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German (L18-1)

Challenge:	Existing MoE designs do not consider computational constraints (e.g., FLOPs, latency) Existing works in MoE consider homogeneous design where the same number of experts of the same size are placed uniformly throughout the network.
Approach:	They propose a framework for designing heterogeneous MoEs under computational constraints.
Outcome:	The proposed framework achieves 4x inference speedup and FLOPs reduction over manual models and within 1 BLEU point of MoE SwitchTransformer over benchmark datasets for NMT.

Copied to clipboard

Pierre-Edouard Honnet, Andrei Popescu-Belis, Claudiu Musat, Michael Baeriswyl

Evaluation of Machine Translation Performance Across Multiple Genres and Languages (L18-1)

Challenge:	Using character-based neural MT, we normalize Swiss German input to address regional diversity.
Approach:	They propose to use character-based neural MT to normalize Swiss German input and phrase-based statistical MT for a low-resource family of dialects.
Outcome:	The proposed system achieves 36% BLEU score when translating from the Bernese dialect.

Copied to clipboard

Marlies van der Wees, Arianna Bisazza, Christof Monz

On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation (2023.findings-acl)

Challenge:	In this paper, we evaluate the impact of genre differences on machine translation (MT) for a diverse set of language pairs . BLEU score differences between genres can be large for all genres and all language pairs.
Approach:	They use multi-genre benchmarks to evaluate the impact of genre differences on machine translation (MT) they train and use genre classifiers to route test documents to the most appropriate genre systems .
Outcome:	The proposed system can improve translation quality for all genres and language pairs .

Copied to clipboard

Liang Chen, Shuming Ma, Dongdong Zhang, Furu Wei, Baobao Chang

Abstract Text Summarization: A Low Resource Challenge (D19-1)

Challenge:	Despite its success, multilingual neural machine translation suffers from the off-target issue, where the translation is in the wrong language.
Approach:	They propose a language-aware vocabulary sharing algorithm that can be used to increase the lexical distance between languages by isolating the vocab of different languages in the decoder.
Outcome:	The proposed algorithm reduces off-target rate for 90 translation tasks from 29% to 8%, while improving overall BLEU score by an average of 1.9 points without extra training cost or sacrificing the supervised directions’ performance.

Copied to clipboard

Shantipriya Parida, Petr Motlicek

DEMETR: Diagnosing Evaluation Metrics for Translation (2022.emnlp-main)

Challenge:	Existing datasets for multilingual text summarization are difficult to construct and lack of human knowledge and language processing abilities in computers makes text summaries a challenging task.
Approach:	They propose an iterative data augmentation approach which uses synthetic data along with the real summarization data for the German language.
Outcome:	The proposed system improves on the development and test sets on the German language text using the state-of-the-art “Transformer” model.

Copied to clipboard

Marzena Karpinska, Nishant Raj, Katherine Thai, Yixiao Song, Ankita Gupta, Mohit Iyyer

Translationese as a Language in “Multilingual” NMT (2020.acl-main)

Challenge:	BLEU scores are based on string overlap, but they are opaque in comparison to newer learned metrics.
Approach:	They propose a dataset to evaluate MT evaluation metrics based on linguistic perturbations in English . they find learned metrics perform substantially better than string-based metrics .
Outcome:	The proposed dataset shows that learned metrics perform better than string-based metrics . the dataset contains 31K English examples that cover 35 different linguistic phenomena .

Copied to clipboard

Parker Riley, Isaac Caswell, Markus Freitag, David Grangier

Low-resource Neural Machine Translation: Benchmarking State-of-the-art Transformer for Wolof<->French (2022.lrec-1)

Challenge:	Recent work examines the impact of translationese in machine translation evaluation using the WMT evaluation campaign.
Approach:	They propose to use a sentence-level classifier to distinguish translationese from original target text to generate a machine translation model that can produce more natural outputs at test time.
Outcome:	The proposed model produces more natural outputs at test time, yielding gains in human evaluation scores on accuracy and fluency.

Copied to clipboard

Cheikh M. Bamba Dione, Alla Lo, Elhadji Mamadou Nguer, Sileye Ba

gaHealth: An English–Irish Bilingual Corpus of Health Data (2022.lrec-1)

Challenge:	Neural machine translation (NMT) systems can translate between French (FR) 1 and Wolof (WO, ISO 639-3), a lowresource Niger-Congo language mainly spoken in Senegal (Gamble, 1950).
Approach:	They propose two neural machine translation systems based on sequence-to-sequence with attention and Transformer architectures to translate between French (FR) 1 and Wolof (WO, ISO 639-3).
Outcome:	The proposed models outperform the classic sequence-to-sequence model in all settings and are less sensitive to noise.

Copied to clipboard

Séamus Lankford, Haithem Afli, Órla Ní Loinsigh, Andy Way

PyMT5: multi-mode translation of natural language and Python code with transformers (2020.emnlp-main)

Challenge:	Existing models for low-resource languages often focus on creating the largest possible dataset for generic translation.
Approach:	They develop a dataset for the specific domain of health for a low-resource English to Irish language pair and compare it to other similar datasets.
Outcome:	The proposed model improved BLEU score by 22.2 points compared with top performing models from the LoResMT2021 Shared Task.

Copied to clipboard

Colin Clement, Dawn Drain, Jonathan Timcheck, Alexey Svyatkovskiy, Neel Sundaresan

UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language (2023.acl-long)

Challenge:	Using Python method text-to-text transfer transformers, developers can easily model source code and natural language.
Approach:	They propose a Python method text-to-text transfer transformer that can translate between all pairs of Python method feature combinations.
Outcome:	The proposed model outperforms similar-sized auto-regressive language models on a large-scale parallel corpus of 26 million methods and 7.7 million method-docstring pairs on the CodeSearchNet test set.

Copied to clipboard

Nuwa Xi, Sendong Zhao, Haochun Wang, Chi Liu, Bing Qin, Ting Liu

Challenge:	Existing studies focus on decoding word-level fMRI volumes from a restricted vocabulary.
Approach:	They propose an open-vocabulary task to bridge fMRI time series and human language . they use a pre-trained language model to construct a robust encoder for cognitive signals .
Outcome:	The proposed task bridges fMRI time series and human language with a baseline model.

Regularized Context Gates on Transformer for Machine Translation (2020.acl-main)

Copied to clipboard

Xintong Li, Lemao Liu, Rui Wang, Guoping Huang, Max Meng

Contrastive Conditioning for Assessing Disambiguation in MT: A Case Study of Distilled Bias (2021.emnlp-main)

Challenge:	Context gates are effective to control the contributions from the source and target contexts in the recurrent neural network (RNN) based neural machine translation.
Approach:	They propose a method to identify source and target contexts and introduce a gate mechanism to control the contributions from source and targets in the advanced Transformer architecture.
Outcome:	The proposed model achieves an averaged gain of 1.0 BLEU score over a strong transformer baseline.

Copied to clipboard

Jannis Vamvas, Rico Sennrich

Enabling Unsupervised Neural Machine Translation with Word-level Visual Representations (2023.findings-emnlp)

Challenge:	Lexical disambiguation is a major challenge for machine translation systems . previous work focused on automatic post-hoc analysis of translations, but rules of what makes a disambiguations correct or incorrect tend to be imprecise.
Approach:	They propose a black-box method that uses contrastive conditioning to detect disambiguation errors.
Outcome:	The proposed method is scalable and reliable for disambiguation evaluations.

Copied to clipboard

Chengpeng Fu, Xiaocheng Feng, Yichong Huang, Wenshuai Huo, Hui Wang, Bing Qin, Ting Liu

LAMBDA: Large Language Model-Based Data Augmentation for Multi-Modal Machine Translation (2024.findings-emnlp)

Challenge:	Unsupervised neural machine translation methods have been observed to make particular errors in comparison to supervised machine translation, such as confusing nouns that pertain to the same semantic category.
Approach:	They propose a method that incorporates images at the word level to augment lexical mappings.
Outcome:	Experiments on a multi-lingual dataset show that the proposed method generates more accurate translations with only monolingual data.

Copied to clipboard

Yusong Wang, Dongyuan Li, Jialun Shen, Yicheng Xu, Mingkun Xu, Kotaro Funakoshi, Manabu Okumura